Papers with multimodal questions
ErrorRadar: Benchmarking Complex Mathematical Reasoning of Multimodal Large Language Models Via Error Detection (2026.findings-acl)
Copied to clipboard
Yibo Yan, Shen Wang, Jiahao Huo, Hang Li, Boyan Li, Jiamin Su, Xiong Gao, YiFan Zhang, Tianlong Xu, Zhendong Chu, Aoxiao Zhong, Kun Wang, Hui Xiong, Philip S. Yu, Xuming Hu, Qingsong Wen
| Challenge: | Current mathematical benchmarks focus on evaluating MLLMs’ problem-solving ability, yet there is a crucial gap in addressing more complex scenarios such as error detection. |
| Approach: | They propose to evaluate multimodal error detection by evaluating two sub-tasks error step identification and error categorization. |
| Outcome: | The proposed task evaluates MLLMs' ability to handle multimodal questions compared to text-only models. |
Benchmarking Deflection and Hallucination in Large Vision-Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks overlook conflicts between visual and textual evidence and the importance of generating deflections when incomplete knowledge is retrieved. |
| Approach: | They propose a dynamic curation pipeline that preserves benchmark difficulty over time . they propose 'vlm-DeflectionBench' benchmark to probe model behaviour under conflicting evidence . |
| Outcome: | The proposed benchmarks overlook conflicts between visual and textual evidence and are prone to obsolescence . the proposed benchmark is based on 2,775 samples spanning diverse retrieval settings . |